CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models

Pre-trained programming language (PL) models (such as CodeT5, CodeBERT, GraphCodeBERT, etc.,) have the potential to automate software engineering tasks involving code understanding and generation. However, these operate in natural channel of code, i.e., primarily concerned with human code. They are not robust changes input thus, potentially susceptible adversarial attacks channel. We propose, Code Attack, a simple yet effective black-box attack model that uses structure generate effective, efficient, imperceptible samples demonstrates vulnerabilities state-of-the-art PL code-specific attacks. evaluate transferability CodeAttack on several code-code (translation repair) code-NL (summarization) across different languages. Attack outperforms NLP achieve best overall drop performance while being more imperceptible, consistent, fluent. The can be found at https://github.com/reddy-lab-code-research/CodeAttack.